Methods, systems and apparatus for controlling movement of transporting devices

ABSTRACT

A system and method for controlling movement of transporting devices arranged to transport containers stored in stacks arranged in a facility having pathways arranged in a grid-like structure above the stacks, the transporting devices being configured to operate on the grid-like structure. A movement optimisation unit is configured to determine a route of a transporting device from one location on a grid-like structure to another location on the grid-like structure for each transporting device. A reservation unit is configured to reserve a path on the grid-like structure for each transporting device based on the determined route. A clearance unit is configured to provide clearance for each transporting device to traverse a portion of the reserved path.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of U.S. patent applicationSer. No. 15/316,249 filed on Dec. 5, 2016, which is a U.S. NationalStage application based on International Patent Application No.PCT/EP2015/062380 filed on Jun. 3, 2015, which claims priority toBritish Patent Application No. 1409883.4 filed on Jun. 3, 2014, theentire content of all three of which is incorporated herein byreference.

The present invention relates to methods systems and apparatus forcontrolling movement of transporting devices. More specifically but notexclusively it relates to storage systems and methods for retrievingunits from a storage system. In particular, but not exclusively, theinvention further relates to systems and methods for coordinating andcontrolling product movement.

Certain commercial and industrial activities require systems that enablethe storage and retrieval of a large number of different products.

One known system for the storage and retrieval of items in multipleproduct lines involves arranging storage bins or containers on rows ofshelves arranged in aisles. Each bin or container holds one or moreproducts of one or more product types. The aisles provide access betweenthe rows of shelves, so that the required products can be retrieved byoperatives or robots that circulate in the aisles.

It will be appreciated, however, that the need to provide aisle space toaccess the products means that the storage density of such systems isrelatively low. In other words, the amount of space actually used forthe storage of products is relatively small compared to the amount ofspace required for the storage system as a whole.

For example, online retail businesses selling multiple product lines,such as online grocers and supermarkets, require systems that are ableto store tens or even hundreds of thousands of different product lines.The supply chains and warehouse operations of these businesses arehighly dependent on their ability to organise, retrieve and return itemsto various containers.

In particular implementations of various warehouse and storage facilitydesigns, containers may be stacked on top of one another and the stacksmay be arranged in rows. The containers may then be accessed from above,removing the need for aisles between the rows and allowing morecontainers to be stored in a given volume or area.

In some of these implementations, the containers are accessed by one ormore robotic or automated means, which navigate through a grid ofpathways to access containers for a variety of different operations,such as moving a container from one location to another for handling,conducting operations upon a container, returning a container to aposition in a warehouse, etc.

The co-ordination of the movement of the one or more robotic orotherwise automated means may be an important consideration indetermining the overall efficiency and scalability of a system forstorage and retrieval of a large number of different products.

In U.S. Pat. No. 6,654,662 and EP 1037828, a storage and retrievalsystem is described wherein “parallelepiped-shaped” containers are“deep-stacked and stock joined” in a vertical framework, forming severalhorizontal layers of containers whose positions at any time are random.A computer system continuously monitors and records the positions of thecontainers; desired containers are retrieved from the top of theframework, unwanted containers being moved, one at a time, and relocatedto temporary locations until the desired container is retrieved, atwhich point the temporarily-relocated containers are returned to thesame stack, and replaced in the same relative order; and the desiredcontainer is (eventually) returned to the top of original stack.

The system described in U.S. Pat. No. 6,654,662 and EP 1037828 hasseveral horizontal coordinate forming layers of containers whosepositions at any time are random. Further, temporarily-relocatedcontainers are returned to the original stack, so that their relativeorder is retained, and wherein the desired container is returned to thetop of the stack.

In contrast to the provided system, it may be advantageous to providedeliberate container placement that may be optimised in the horizontaland the vertical domains based on different criteria such as trafficflows, frequency of access (historic, current and forecast), specificcontainer groupings, access time, fire-resistance and or otherenvironmental partitions.

There is need, therefore, for systems and processes for coordinating andcontrolling product movement.

According to exemplary embodiments, a system is provided for controllingmovement of transporting devices arranged to transport containers, thecontainers being stored in stacks arranged in a facility, the facilityhaving pathways arranged in a grid-like structure above the stacks, thetransporting devices being configured to operate on the grid-likestructure, the system comprising: a movement optimisation unitconfigured to determine a route of a transporting device from onelocation on a grid-like structure to another location on the grid-likestructure for each transporting device; a reservation unit configured toreserve a path on the grid-like structure for each transporting devicebased on the determined route, wherein the path reserved for eachtransporting device is provided such that no two transporting deviceshave locations on the grid-like structure which would cause transportingdevices to overlap at a same time; and a clearance unit configured toprovide clearance for each transporting device to traverse a portion ofthe reserved path.

A method is also disclosed for controlling movement of transportingdevices arranged to transport containers, the containers being stored instacks arranged in a facility, the facility having pathways arranged ina grid-like structure above the stacks, the transporting devicesconfigured to operate on the grid-like structure, the method comprising:determining a route from one location on the grid-like structure toanother location on the grid-like structure for each transportingdevice; reserving a path on the grid-like structure for each of thetransporting devices based on the determined route, wherein the pathreserved for each transporting device is provided such that no twotransporting devices have locations on the grid-like structure whichwould cause transporting devices to overlap at a same time; andproviding clearance for each transporting device to traverse a portionof the reserved path.

According to some embodiments of the invention, a system is provided forcontrolling the movement of one or more of transporting devicestransporting a plurality of objects and operating in a facility having aplurality of pathways, the system comprising:

-   -   one or more computers configured to execute computer        instructions which when executed provide:    -   one or more utilities determining and reserving routes for        moving the one or more transporting devices through the        plurality of pathways; and    -   one or more utilities providing a clearance system for        permitting or stopping movement of the one or more transporting        devices to avoid collisions.

In some embodiments, the system further comprises one or more utilitiesconfigured to optimise the movement and actions of the one or moretransporting devices through the plurality of pathways.

In some embodiments, the system further comprises one or more utilitiesconfigured to optimise the placement of the plurality of objects by theone or more transporting devices in the facility.

In some embodiments, the system further comprises one or more utilitiesthat control movement or operations conducted within one or moreworkstations.

In some embodiments, the one or more utilities that optimise themovement and actions utilize pathfinding algorithms.

In some embodiments, the one or more utilities optimise the movement andactions using congestion mitigation techniques.

In some embodiments, the one or more utilities optimise the movement andactions using machine learning techniques.

In some embodiments, the one or more utilities optimise the placement ofthe plurality of objects within the facility, and update stock levels ofthe facility based on the placement of one or more of the plurality ofobjects within the facility.

In some embodiments, the one or more utilities control the movement ofthe one or more of transporting devices based at least on throughput ofthe one or more workstations.

In some embodiments, the one or more utilities provide a clearancesystem configured to be tolerant to missed communications with at leastone of the one or more transporting devices.

In some embodiments, the one or more utilities determine and reserveroutes, and instruct a plurality or transporting devices to cooperate intransporting one or more of the plurality of objects.

In some embodiments, the one or more utilities determine and reserveroutes to move idle transporting devices from otherwise optimal routesfor other transporting devices.

In some embodiments, one or more of the plurality of objects are storedwithin a plurality of containers.

In some embodiments, the facility stores the plurality of containers ina plurality of stacks.

In some embodiments, the one or more utilities determine and reserveroutes for transporting one or more of the plurality of containers.

In some embodiments, the one or more utilities determine and reserveroutes to control one or more of the plurality of transporting devicesto retrieve one or more of the plurality of containers from within oneor more of the plurality of stacks.

In some embodiments, retrieving the one or more containers from withinthe one or more stacks further requires moving one or more othercontainers in the stack prior to accessing the one or more containersfor retrieval.

In some embodiments, moving the one or more other containers comprisesplacing each of the one or more other containers in an optimisedposition within the facility.

In some embodiments, the facility is divided into a plurality ofsub-grids to reduce processing complexity.

In some embodiments, one or more control commands to control themovement of the plurality of transporting devices is provided to theplurality of transporting devices in advance of the movement of theplurality of transporting devices.

In various aspects, the disclosure herein provides methods, systems, andcorresponding machine-executable coded instruction sets for coordinatingand controlling product movement in at least a semi-automated orderfulfillment system comprising a holding facility. In various aspects,the disclosure provides improvements in the coordination and control ofthe movement of robots handling a variety of goods in fulfillment oforders which, in some instances, may include a variety of items havingdifferent sizes, weights, fragilities and other characteristics.

In various embodiments of the above and other aspects, such a holdingfacility can include one or more storage apparatuses. In the same orother embodiments, at least a portion of the holding facility may beconfigured to move dynamically.

In the same or other embodiments, the holding facility may be sharedbetween two or more workstations.

The invention will now be described with reference to the accompanyingdiagrammatic drawings, the drawings being to be exemplary and notlimiting, and in which like references are intended to refer to like orcorresponding parts.

FIG. 1 is an illustrative diagram providing a generic computer hardwareand software implementation of certain aspects, as detailed in thedescription.

FIG. 2 provides a sample block diagram of the system, according to someembodiments of the invention.

FIG. 3 provides a sample block diagram of the robot control system inmore detail, according to some embodiments of the invention.

FIG. 4 provides a sample workflow for a sample recursive path-findingalgorithm, according to some embodiments of the invention.

FIG. 5 provides a sample heat map, according to some embodiments of theinvention.

FIG. 6a provides a table demonstrating how a search branch's projectedcost could change as a result of using different coefficients in thepath search algorithm. These coefficients could be derived usingrefinement by a machine learning technique according to some embodimentsof the invention.

FIG. 6b illustrates how the use of different cost coefficientsdemonstrated in FIG. 6a changes which branch will be selected by thenext iteration of the search algorithm.

FIG. 7 provides a sample perspective view of a warehouse, according tosome embodiments of the invention.

FIG. 8 provides a sample diagram of a robot with a winch and acontainer, according to some embodiments of the invention.

Preferred embodiments of methods, systems, and apparatus suitable foruse in implementing the invention are described through reference to thedrawings.

Fully- and semi-automatic goods storage and retrieval systems, variousaspects of which may sometimes be referred to as “order fulfillment,”“storage and retrieval,” and/or “order picking” systems, can beimplemented in a wide variety of types and forms. One manner ofproviding access to goods stored for fully- and/or semi-automaticretrieval, for example, comprises placement of goods, which may be ofany desired type(s), in bins or other containers (hereinafter referredto generically as containers), and stacking and/or otherwise disposingthe containers in racking or vertically in layers, such that individualcontainers may be accessible by wholly or partially-automated containerretrieval systems.

In some embodiments, the systems may include systems beyond goodsstorage and retrieval, such as systems where goods are processed,repaired, manipulated, assembled, sorted, etc., and the movement ofgoods, products, parts, components, subcomponents is required, bothwithin a facility and/or to other facilities or transportation.

For the purposes of this specification, a storage facility for thestorage, retrieval, processing and/or fulfillment of orders, whereinaccess to such goods is provided by fully or semi-automatic retrieval,is referred to as a “hive”. The “hive” may be comprised of a grid-likelayout of the potential pathways for the movement of robotic elements ordevices (“robot”) to traverse and perform operations at variouslocations in the “hive” (referred to as the “grid”).

The specification is not limited to only systems that have “hives”,“grids”, and/or “robots”, but systems that broadly control and/orcoordinate the movement and/or activities of a plurality of devices mayalso be contemplated. These devices may be configured for thetransportation of various items, such as goods and/or products, and/orcontainers that may be empty and/or holding such goods and/or products.These devices may further be involved in the fulfillment of orders butmay also be involved in any other type of activity, such as transportingcontainers to and from workstations, moving objects from sourcelocations to target locations, etc.

As indicated, the devices may be robots, and the devices may beconfigured to move around a hive, and/or communicate with a controlsystem to coordinate/receive instructions on their movement. In someembodiments, the devices may be configured to communicate amongstthemselves, and/or coordinate movement amongst themselves. Accordingly,the devices may have various transporting means, communications means,powering means, processing means, processor means, sensor means,monitoring means, on-board workstations, electronic/physical storagemeans and/or lifting/transporting means (such as a winch, arms, etc.).

While the devices may be configured to receive instructions from thesystem, there may be situations where the devices lose communicationswith the system, have degraded communications pathways and/or do notreceive communications from the system within a particular time frame.

In some embodiments, the devices may also be configured to communicateamongst each other, and/or sense the presence of each other. Thesecommunications and/or sensory inputs may be utilized, for example, incrowdsourcing information about the environment, providing redundantcommunications channels, verifying instructions, etc.

Fulfillment of orders may include various operations, such as, but notlimited to: assembling orders where various products are purchased andaggregated for delivery to a customer, such as for a grocery chain;assembling products with various subcomponents; conducting variousoperations on products (such as soldering components together), sortingproducts, etc.

Orders may also be returned, for example, if an order is cancelled, adelivery fails, etc. In some scenarios, while an order is in the processof fulfillment within the hive, it may be cancelled and the productitems may need to be returned. In some scenarios, the items may need tobe placed again into containers, and the containers moved to variouslocations. In some scenarios, a workstation may need to conduct tasks toreject/rework products when an order is returned or cancelled.

Furthermore, as mentioned above, individual containers may be invertical layers, and their locations in the “hive” may be indicatedusing co-ordinates in three dimensions to represent the robot or acontainer's position and a container depth (e.g. container at (X, Y, Z),depth W). In some embodiments, locations in the “hive” may be indicatedin two dimensions to represent the robot or a container's position and acontainer depth (e.g. container at (X, Y), depth Z).

The “hive” itself may be a “dynamic” environment, in the sense thatrobots and workstation locations may be associated with different partsof the hive for engaging in actions. For example, robots may need toaccess a specific container in a specific location in the hivedimensions (e.g. container at (X, Y, Z), depth W) to fulfil a particularorder or to store a product in the “hive”. This involves movements ofthe robots along various possible paths, for example, along the top ofthe grid, and then accessing particular containers at selected depths ofa stack.

The access of particular containers at selected depths of a stack maynecessitate the movement of containers which may otherwise obstruct theability to access a particular container (e.g. where the containers arestacked, a number of containers must be moved first to be able to accessa container that is not at an accessible end of the stack). In someembodiments, it may be advantageous to have the system configured toprovide for the evaluation and optimisation of a new position for everycontainer that has to be removed to access a target container.

Containers moved off of a stack need not be moved back to their originalstack.

One of the potential advantages is the ability to modify thedistribution of containers such that the containers are located in moreeasily accessible or otherwise more convenient locations.

This may help maintain an optimal distribution of containers within thefacility, for example, biasing containers that are expected to be inhigher demand in more easily accessible locations, such as locationsnearby or within workstations, to reduce travel distance.

FIG. 7 provides a sample perspective view of a warehouse, according tosome embodiments of the invention.

Robots may have various shapes, sizes and configurations, and may havevarious communications means, sensors and tools. In some embodiments,each robot may be able to communicate with the control system through aset of frequency channels established through a set of base stations andbase station controllers. Robots may utilize various tools to move andobtain containers from a stack, including, for example, a winch to carrya container.

FIG. 8 provides a sample diagram of a robot with a winch and acontainer, according to some embodiments of the invention.

The grid is not limited to rectangular grid elements and may becomprised of curved tracks, tracks up and down, etc. The grid pathwaysmay have intersections and may be accessed by more than one robot.

Each grid may be segmented, physically or logically, into one or moresub-grids.

The grid may be comprised of one or more workstations. Workstations maybe manual, semi-automated or fully automated, and may consist oflocations or areas where operations are conducted within the hive, oroperations are conducted in relation to the hive, containers orproducts, such as, moving products in or out of the hive, manufacturingproducts, assembling products, processing products to their components,providing staging locations to support other steps or operations, etc.

Workstations could include, for example, locations where items are movedfrom inbound carriers, locations where products have various operationsconducted on them (e.g. assembly of components, painting, sorting,packaging, disassembly, reworking products, fixing packaging, replacingproducts in cancelled orders, rejecting returned products, disposingproducts), products are moved to outbound carriers, locations withcapabilities for refrigeration, locations where components or objectsare assembled, locations used for staging or pre-fetching products,locations where robots are repaired and maintained, locations whererobots are charged, locations where workers “pick” products to be placedinto containers, locations where workers “pick” products to be removedfrom containers in fulfillment of orders, bags are placed intocontainers, etc.

Where items/products are returned to the hive, the system may supportand/or control the process of bringing back the product, reworking theproduct, and/or disposing the product if rejected. The scenario may, insome embodiments, involve processing the returned container (which maybe a delivery tote or other object as well) at a workstation todetermine whether it can be accepted back into the system, whether itneeds reworking/repackaging, and/or whether the product should bedisposed of instead (e.g. a perishable product has expired).

Workstations may have one or more workers or robots present to conductvarious tasks, such as picking items for fulfillment of orders.

In some embodiments, workstations may also be stations with conveyors,refrigerators, various tooling technologies and/or other technology tomanipulate, paint, fasten, repair, freeze, heat, expose to chemicals,refrigerate, filter, assemble, disassemble, sort, package, scan, test,transport, store or process goods, containers, etc.

The workstations may have their own pathways within the facility, sharepathways with the facility, etc. The workstations may also have variousinput and output pathways or other types of entry/egress points withinthe facility.

In some embodiments, the workstations communicate with one or morewarehouse management systems to provide information and data related tothe status of the workstation, workflow, required containers, issues,status of products held or otherwise manipulated (e.g. sub-componentsbeing assembled together), etc.

Upon receipt of an order from a customer for multiple items stored in astorage and retrieval system, fully or semi-automated container handlersmay retrieve storage containers containing relevant items from a grid,racking, or other ordered arrangement of storage containers, and deliverthem to one or more workstations.

At the workstations, items may be removed from the storage containersand placed in an intermediate holding facility before being picked intodelivery containers.

As indicated throughout this specification, it may be advantageous tohave a distribution of containers such that containers that are likelyinputs for certain workstations are located close in proximity to thoseworkstations.

The picking of items from containers may be done manually by humanpickers, or be conducted in a semi-automated or fully-automated mannerwith the assistance or participation by various mechanical or roboticelements.

Where individual containers are stacked vertically in layers, accessinga container that is not at the top layer may require a further set ofoperations to move containers stored at layers above the desiredcontainer prior to being able to access the desired container. Forexample, if a desired container is one level below the top layer, thecontainer residing within the top layer may need to be moved to anotherlocation prior to accessing the desired container. As indicatedthroughout this specification, it may be advantageous to have adistribution of containers such that containers holding items in greaterdemand are biased towards the more easily accessible positions (e.g. ifa vertical stack of containers, the uppermost levels). In someembodiments, an optimisation module provides an optimal location forthese containers to be positioned.

In a typical commodities picking operation adapted for handling a largevariety of items, such as a grocery order processing system, it issometimes found that a wide range of items of various sizes, shapes,weights, and other characteristics must be handled or otherwiseaccommodated, and that these items may need to be moved around afacility to various stations for various operations to be conducted,during the fulfillment of one or more order(s). Depending on the size,organisation and arrangement of the facility, the movement of theseitems may be advantageously optimised so that items are movedefficiently, collisions are avoided and dependencies are resolved (e.g.objects are picked and unloaded in a proper order).

The various actions involved in these working groups such as placementof items in different types of containers, storage of containers withinthe hive, and sorting/delivery as described can result in variousmovements of the containers, including within the hive, but alsoplacement of containers in different locations. These actions mayinclude inbound/outbound activity to the hive (e.g. bringing items toand from a warehouse), and may also include associated sorting/deliverysystems such as management of conveying product off vehicles forstoring/delivery, and also conveying onto vehicles for orderfulfillment.

In other embodiments of the invention, the workstations may be utilizedto provide other types of item handling or container handling, such asworkstations where actions are performed on items (e.g. assembly,disassembly, modification, sorting, drying, freezing, testing, chemicalexposure, physical manipulation, fastening, repair, printing, painting,cutting, packaging, storage, processing, welding, tempering,reprocessing). These workstations may be manual, semi-automated, orautomated and have various parameters associated with their operation orperformance, such as throughput, required inputs, required outputs, loadbalancing, required delays (e.g. drying time), etc.

The various movements of the robots, placement of containers/objects,and control over selecting when to remove product from containers may becontrolled and optimised by one or more control systems. This controlmay include the implementation of various strategies, including, forexample:

-   -   The central management of one or more robots;    -   Managing the robots not only to retrieve containers for        processing, but also to “pre-fetch” containers to more        convenient locations, for example, locations close to or within        workstations;    -   Path optimisation through the application of one or more path        finding algorithms (e.g. branch and bound techniques);    -   Path optimisation through the application of one or more        heuristic techniques to the one or more path finding algorithms        (e.g. projected lowest cost, where in one embodiment of the        invention, cost may be calculated as a function of total time to        target, and total “accumulated heat” for congestion mitigation        on the grid when viewed as a whole);    -   Scheduling/pre-processing movement pathways in advance;    -   Conducting simulations to determine optimal depth of analysis in        scheduling/pre-processing movement pathways;    -   Application of machine learning techniques to optimise exponents        and coefficients applied to the cost functions of one or more        algorithms. In some embodiments, this may include path finding,        container placement and workstation load-balancing algorithms;    -   A just-in-time robot path conflict resolution system;    -   The subdivision of the robot population into separate control        groups each containing one or more robots;    -   Container location optimisation through various means, such as        the application of a scored selection algorithm using weighted        cost coefficients;    -   Optimisation of the allocation of work to workstations;    -   Pre-processing of tasks and actions to schedule future tasks and        actions.

In an example environment where one or more robots are used for thefully or semi-automated retrieval, storage and movement of objects, thewarehouse may first have to decide how to distribute the container loadsbetween different robots or groups of robots.

For example, the allocation of tasks across various groups of robots andpick stations may be optimised depending on the particular layout of awarehouse, the placement of items, particular characteristics of goods(e.g. the item is expiring or is potentially dangerous) and the workflowordering.

For example, it may be desirable to have a system that intelligentlycompares potential paths for a robot to take to its destination, takinginto consideration, among others, the potential congestion along thatpath, the time required to complete operations, the potential forcollisions, the objects held in the inventory of a particular robot,predicted future operations and the characteristics of a particularrobot (e.g. battery levels, service issues).

It may further be desirable to have a system that intelligently adaptsto various conditions in the “hive”, such as idle robots that may hinderor block the path of a robot, obstacles, or other robots reserving pathsthat a particular robot is seeking to traverse.

It may further be desirable to have a system that intelligentlypositions containers based on an algorithm to bias the system towards adistribution of containers that aids in the efficient retrieval of items(e.g. containers with items of high-demand SKUs are kept near the topand near their workstations for ease of access).

These are non-limiting examples, and any optimisation methods,arrangements or considerations may be implemented.

Referring to FIG. 2, a schematic diagram is provided of sample fully-and semi-automatic goods storage and retrieval systems, according tosome embodiments of the invention. FIG. 2 is provided at a high level,illustrating an example implementation. Various implementations of thesystem 200 may involve more or less components and FIG. 2 is merelyprovided as an example.

The system 200 is comprised of a robot control system 202; amaintenance/monitoring system 204; a base station controller 206; one ormore base stations 208 a and 208 b; one or more robots 210 a, 210 b and210 c, and one or more charger stations 230. While only two basestations 208 a and 208 b, and three robots 210 a, 210 b and 210 c areillustrated, it should be understood that that there may be more, orless robots and base stations in other embodiments of the system.

There may be one or more warehouse management systems (WMS) 232, ordermanagement systems 234 and one or more information management systems236.

The warehouse management systems 232 may contain information such asitems required for an order, SKU#s in the warehouse, expected/predictedorders, items missing on orders, when an order is to be loaded on atransporter, expiry dates on items, what items are in which container,whether items are fragile or big and bulky, etc.

In some embodiments, the warehouse management systems 232 may be incommunication with the workstations and may also contain informationrelated to the operation of the workstations, such as the status of theworkstation, what products and/or what containers the workstation isrequired to receive at particular times, what products and/or containersthe workstation will be required to have moved to another location atparticular times, the expected workflow for operations at theworkstation, the number of robots currently waiting to bring containersto a workstation, etc.

The robot control system 202 may be configured to control thenavigation/routing of robots, including, but not limited to, moving fromone location to another, collision avoidance, optimisation of movementpaths, control of activities to be performed, etc. The robot controlsystem 202 may be implemented using one or more servers, each containingone or more processors configured based upon instructions stored uponone or more non-transitory computer-readable storage media. The robotcontrol system 202 may be configured to send control messages to one ormore robots, receive one or more updates from one or more robots, andcommunicate with one or more robots using a real or near-real timeprotocol. The robot control system 202 may receive informationindicating robot location and availability from one or more basestations 208 a and 208 b. The robot control system 202 may be configuredto keep track of the number of robots available, the status of one ormore robots, the location of one or more robots and/or their currentinstruction sets. The robot control system 202 may also be configured toprocess and/or send control messages to the one or more robots inanticipation of future movements, potentially reducing the processorload, and also proactively managing the traffic load with respect to thecommunications links. Such an implementation could be advantageous inlight of complex algorithms in use by the robot control system 202 indetermining optimal pathways, calculating optimal locations forcontainers and/or determining reservations and/or clearances.

In some embodiments, the servers may utilize a ‘cloud computing’ typeplatform for distributed computing. A cloud-based implementation mayprovide one or more advantages including: openness, flexibility, andextendibility; manageable centrally; reliability; scalability; beingoptimized for computing resources; having an ability to aggregateinformation across a number of users; and ability to connect across anumber of users and find matching sub-groups of interest. Whileembodiments and implementations of the present invention may bediscussed in particular non-limiting examples with respect to use of thecloud to implement aspects of the system platform, a local server, asingle remote server, a software as a service platform, or any othercomputing device may be used instead of the cloud.

In some embodiments, the movement optimisation module may utilize one ormore control groups to segregate robots into the one or more groups. Theuse of control groups for large grids may provide certain advantages,such as the ability to maintain operation of a very large grid wheneverreal-time computation cannot keep up with re-planning after a controlanomaly such as when (i) wireless communication link drops moresequential packets than allowed for in planning; (ii) one or more robotsfail; (iii) one or more robots operate outside the pre-determinedtolerance on performance.

A control stop message may be broadcasted to the robots in a particular“control group”; potential benefits from broadcasting messages asopposed to sending individual messages may include improvedcommunications through the use of multiple transmission slots and apotentially higher signal to noise ratio.

In some embodiments, the robot control system 202 may be configured todynamically assign robots to different “control areas” as they moveacross the grid.

The maintenance/monitoring system (MMS) 204 may be configured to providemonitoring functions, including receiving alerts from one or more robotsor one or more base stations, establishing connections to query robots.The MMS 204 may also provide an interface for the configuration ofmonitoring functions. The MMS 204 may interact with the robot controlsystem 202 to indicate when certain robots should be recalled.

The base station controller 206 may store master routing information tomap robots, base stations, and grids. In some embodiments, there may beone base station controller 206 per warehouse, but in other embodimentsof the invention, there may be a plurality of base station controllers.The base station controller 206 may be designed to provide highavailability. The base station controller may be configured to managedynamic frequency selection and frequency allocation of the one or morebase stations 208 a and 208 b.

The base stations 208 a and 208 b may be organised as a pool of basestations, which may then be configured to be active, on standby or tomonitor the system. Messages may be routed through a variety ofcommunications means to/from robots. The communications means may be anycommunications means, in some embodiments, the communications means maybe a radio frequency link such as those falling under wireless standard802.11. The base stations 208 a and 208 b may further include processingunits 212 a, 212 b, digital signal processors 214 a, 214 b, and radios216 a, 216 b.

The one or more robots 210 a, 210 b, and 210 c may be configured to movearound the grid and to perform operations. Operations may include movinga container from one stack to another, going to a charging station torecharge, etc. The one or more robots may be assigned to communicatewith the one or more base stations 208 a and 208 b.

The one or more robots 210 a, 210 b, and 210 c may not all be the sametype of robot. There may be different robots with various shapes,designs and purposes, for example, there may be a robot with a footprintof a single grid position which winches containers for internal stowage,a cantilever robot, a bridge robot, a recovery robot, etc.

In some embodiments, the one or more robots 210 a, 210 b and 210 c havewinches on them which may be used to retain a container for movementfrom one position on the grid to another.

The robots 210 a, 210 b and 210 c may have, respectively, radios 218 a,218 b, 218 c, digital signal processors 220 a, 220 b, 220 c, processors222 a, 222 b, 222 c, real time controllers 224 a, 224 b, 224 c,batteries 226 a, 226 b, 226 c and motors, sensors, connectors, etc., 228a, 228 b, 228 c.

The one or more charger stations 230 may be configured to provide powerto charge batteries on the one or more robots 210 a, 210 b and 210 c.The one or more charger stations 230 may further be configured toprovide high speed, wired data access to the one or more robots, andseveral charge stations may be placed around the grid for use by the oneor more robots 210 a, 210 b and 210 c.

Referring to FIG. 3, a block diagram is provided of the control system202, according to some embodiments of the invention. The block diagramis provided for illustrative purposes to identify some of the componentsof control system 202 in more detail, however, not every module orinterface identified may be required and, in various embodiments, moreor fewer modules may be included.

The control system 202 may be configured to evaluate how to improve workallocations, movements of product and placement of product. According tovarious embodiments of the invention, optimisations may be run in realtime, while others are run, for example, periodically during down timeor less active times.

The control system 202 may be configured to schedule when specific typesof movements should happen and in what order they should occur,depending on the application of various business rules, indicatingpriority, etc. The control system 202 is configured to determine bothinbound and outbound factors in making decisions even relative toproduct placement for example. For example, the control system 202 mayact on estimated delivery location of product supply, and estimatedoutbound delivery of product. The control system may make decisions, andmay send signals for execution by an automatic system, and/or mayallocate tasks efficiently to humans (pickers, loaders etc.).

The control system 202 may determine that one or more robots or one ormore pickers should conduct one or more actions in the fulfillment of anorder or for any other purpose. The action of the one or more robots mayrequire the robots to traverse the grid, and/or to conduct actions, suchas retrieving a container.

The control system 202 may be configured to analyze various pathways inthe grid to determine one or more paths that may potentially bepreferential relative to other pathways, given a set of constraints andconditions. These preferential pathways may then be provided, one-time,periodically and/or dynamically to the robots to control their movementsthroughout the grid.

A path may be preferential for a number of reasons, including, but notlimited to: less distance travelled, greater expected average velocityof robot, lower probability of encountering traffic (i.e. congestion),less total time required, lower probability of collision, less powerused, ease of switching to alternate pathways, ability to avoidobstacles (e.g. a broken robot, a dropped item, a broken path, a part ofthe path is under repair).

The control system 202 may use various algorithms to identify, designand/or control the movement of various robots it is connected to. Insome embodiments, the control system is implemented using one or moreservers, each containing one or more processors configured to performone or more sets of instructions stored upon one or more non-transitorycomputer readable media. Potential advantages for computerimplementation include, but are not limited to, scalability, ability tohandle large amounts of processing and computational complexity,increased reaction speed, ability to make decisions quickly, ability toconduct complex statistical analysis, ability to conduct machinelearning, among others.

These algorithms are discussed more at depth further in thespecification, and sample path-finding algorithms and heuristicapproaches are provided, according to some embodiments of the invention.

Constraints may include the current layout of the grid, the physics ofthe robot (e.g. maximum velocity, turning radius, turning speed, maximumacceleration, maximum deceleration), congestion (e.g. expected trafficload at a certain pathway or intersection), established ‘highways’,impact of objects being carried by the robot (e.g. big, bulky, orfragile objects), robot status and condition (including batterycondition, damage, maintenance issues), and station status (e.g. thedestination station is full or temporarily blocked).

The control system 202 may be a real or near-real time control system(controlling the actions of the various units including robots andoptionally the associated other units involved such as conveyors,pickers, humans, etc.). The control system 202 may be comprised of oneor more modules. The one or more modules may include a control interface302, a movement optimisation module 304, a product placementoptimisation module 306, a robot physics model module 308, a businessrules module 310, a clearance module 312, a reservation module 314, acommand generation and scheduler module 316, a robot communicationsmodule 318, a charge manager module 320 and an alert/notification module322. These modules may be implemented in various ways, in someembodiments they are implemented as applications stored as instructionson one or more computer-readable media to be performed by one or moreprocessors.

The control system 202 may provide real or near-real time control of theallocation of work, workstation operations, the movement of robotsand/or the placement of containers, according to some embodiments of theinvention. The allocation of work, movement and placement of containersmay be precipitated by actions as relevant to activities within awarehouse, such as the fulfillment of orders, the redistribution ofcontainers to more easily accessible positions, estimated dispatchsequences, maintenance operations, workstation operations, anticipationof future orders, etc.

The control interface 302 provides an interface for various externalsystems to provide directions and information into the control system202. The control interface 302 may, in various embodiments, provideinterfaces for human users and/or interfaces for interfacing withvarious machines and systems.

Interfaces for humans may include, for example, keyboards, visualdisplays, command prompts, etc.

Interfaces for machines and systems may include application programmableinterfaces (APIs), implemented using different specifications,including, but not limited to, simple access object protocol (SOAP) andrepresentational state transfer (REST) services, and/or interfaceswritten in various programming languages. The control interface 302 mayinteract with various external databases, including but not limited tovarious warehouse management systems and order management systems; andalso may receive information from the various robots (e.g. a robot ismalfunctioning, a robot requires charging, a robot is en route to thedestination, a robot has encountered an unexpected obstacle, etc.).

The control interface 302 may also receive and transmit information toand from the warehouse management system (WMS) relevant to the controland movement of robots and containers. Such information may include, butis not limited to, grid location and sizing, the establishment ofsub-grids, master records of inventory and orders, locations ofworkstations, parameters related to workstations, and/or also thedispatch sequence (e.g. when orders need to go out). As actions areperformed, containers brought to workstations, workstation operationscompleted, delivery totes filled, etc., the control interface 302 mayprovide updates to the WMS. In some embodiments, there is a confirmationprocess between the WMS and the control interface 302. These updates tothe WMS may include, for example, updated stock levels related toparticular SKU#s, updated container positions, updated object positionswithin containers, updated facility conditions, etc.

In some embodiments of the system, in addition to the WMS, there mayalso be a separate order system that contains and provides informationregarding various orders entering the system, the fulfillment of orders,workstation operations, upcoming orders and predicted orders.

The control interface 302 may also receive commands to stop theoperation of a particular robot, a group of robots or all of the robots(e.g. in the event of a malfunction, an emergency, etc.).

The movement optimisation module 304 may be configured to optimise themovement of robots through applying various algorithms to determinepotentially advantageous routes from one location to another. Thepotential advantages may include shorter distance travelled, lowerlikelihood of encountering congestion, shorter time required, lowerpower consumption, co-ordination with movements of other robots, routingaround obstacles such as broken robots or broken areas of track,co-ordination with various workstation operations, etc.

The movement optimisation module 304 may be configured to provide workallocation, planning and scheduling functions, including developing aset of tasks and then selecting which pick station or robot shouldconduct which task. For example, this may be based upon where a robot ora pick station is located, the particular capabilities of the robot orpick station, etc. Further, the particular permutations and set ofactions required to fulfill a particular order are determined andactions/tasks are developed for one or more robots and or one or morepick stations. Functions may include, among others, delivering emptycontainers to inbound stations, placing containers loaded with goodsaround the warehouse, bringing containers to pick stations or otherareas, moving containers from one location in the warehouse to another,etc.

The movement optimisation module 304 may be configured to interact withthe product placement optimisation module 306 in determining a set ofpotentially advantageous locations to place an object. For example,given that a container containing items of a particular SKU# that isrequired at a high frequency, the product placement optimisation module306 may indicate that it should be placed at a certain location in acertain stack that is more accessible for retrieval. Conversely, if acontainer contains items of a particular SKU# that is required at a lowfrequency, the product may be determined to be placed at a lower depthwithin a less easily accessible grid location.

In optimizing movement, the movement optimisation module 304 may beconfigured to consider various factors involved in both movement and theperformance of an operation, such as the expected time required to getto a particular location, how deep the container is within a stack, howlong it would take to dig a container out of a stack, the variousoperations necessary to move containers located above to otherlocations, etc.

The movement optimisation module 304 may also be provided a set ofinputs from the robot physics model module 308, which may communicate aset of constraints on the movement of the robot depending on variousfactors (e.g. the robot may only move at 50% of the maximum velocity asthe robot is currently carrying delicate objects). The movementoptimisation module 304 may coordinate the movement of boxes into thegrid, out of the grid and within the grid.

In some embodiments, the movement optimisation module 304 maydynamically recalculate preferential paths during the course of arobot's journey to potentially determine an updated set of paths asconditions and constraints change over time.

In some embodiments, the grid may be pre-processed by the movementoptimisation module 304 to potentially increase processing speed and/orreduce processing load. Other methods of reducing processing load arealso contemplated, such as reducing the depth/breadth of searching,dividing the grid into sub-grids, distributed processing, caching futureroutes, computationally simplifying the grid (e.g. reducing the numberof nodes under analysis), reducing path re-calculation, etc.

In some embodiments, the movement optimisation module 304 may divide agrid into a plurality of smaller sub-grids for analysis. Such divisionmay ease demands on processing power, which may be particularly usefulif grid sizes are very large; for example, a 1000×1000 grid may bebroken into 100 100×100 grids and each grid analyzed separately. Thismay be further useful if the system is attempting to control a verylarge number of robots or take into consideration a large number ofconditions.

The movement optimisation module 304 may also interact with theclearance module 312 and the reservation module 314 in determiningwhether the navigation of a proposed pathway will encounter issuesinvolving the clearances and reservations of other robots and alsodetermining pathways that may reduce the chances of encountering theseissues.

Where a desired container is located within a stack at various depthswithin the stack, the movement optimisation module 304 may be requiredto control one or more robots in the movement of containers off thestack so that the desired container is accessible. The movementoptimisation module 304 may coordinate movement across one or morerobots such that the one or more robots cooperate in moving containersoff the stack.

In some embodiments, the movement optimisation module 304, it may not benecessary or even desired to replace the containers on the stack,rather, containers that have been moved off of the stack may have anoptimal position determined by the product placement optimisation module306, and may be moved there by the one or more robots. A potentialadvantage of such an embodiment is that increased efficiency may befound when containers are not replaced in their original position butrather placed in a more optimal position.

In some embodiments, the clearance module 312, the reservation module314 and the movement optimisation module 304 are utilized together as apath conflict resolver, wherein a movement optimisation module 304develops a path and then reserves the path using the reservation module314, and the clearance module 312 provides a just-in-time approach todetermining priority when robots are engaged in potentially conflictingpaths.

In some embodiments, the movement optimisation module 304 is configuredto account for situations where a robot is attempting to take acontainer to a full station. In this situation, the movementoptimisation module 304 is configured to instruct the robot to take thecontainer nearby to be held by the robot until the station can acceptthe container. When the station can accept the container, it is droppedoff. In these embodiments, held containers may be dropped off inpriority order.

In some embodiments, if a station becomes free for drop off before theheld container arrives at its holding location, the movementoptimisation module 304 will re-plan to drop off the container directly,without holding.

In some embodiments, the movement optimisation module 304 is furtherconfigured for pre-fetching operations, wherein a container is movedcloser to a station prior to it being required at the station.Containers are then ready to be dropped off when required, which mayreduce the uncertainty of the drop off time.

In some embodiments, the system may be configured to plan the robotpaths and establish robot path reservations sufficiently far in thefuture to allow the algorithms to complete.

In some embodiments, the degree of forward planning required may becomputed by simulation.

The simulations may be used to adjudicate (in a statistical sense)between the efficiency gains of planning far into the future; againstthe efficiency loss of having higher probabilities of having to re-planwhen robots fail to maintain their plan, because of various reasons,such as short-term communication packet losses, and/or robots runningoutside the allowed tolerances.

The product placement optimisation module 306 may be configured todetermine a set of potentially advantageous locations to place aparticular container containing particular items. The product placementoptimisation module 306 may utilize relevant information, such as thelayout of the grid, the frequency at which a particular item isrequested, future ordering, predicted future ordering, the location ofworkstations, the location of charging stations, and the congestionlevel of particular areas and branches of the path, among others, indetermining the set of potentially advantageous locations to place aparticular container containing particular items.

A robot may be tasked with transporting one or more containers, each ofwhich may contain one of more items, to satisfy the demands of one ormore service pick stations. Each container may be provided with an indexnumber related to a stack location so that particular containers may bebiased towards particular stack locations.

The distribution of and positioning of a container within a stack orrelative to the layout of a grid may contribute towards the overalloperational efficiency of a warehouse.

There are various situations where a container needs to be placedsomewhere in the facility and these situations present an opportunity tore-evaluate and/or optimise the distribution of the position ofcontainers in the facility.

There are various considerations that would indicate that a position isbetter or worse than another position, which may include, but are notlimited to, distance to workstations, level of congestion in the area,which containers will be blocked by a particular container, logicalgroupings of containers relative to environmental factors (e.g.containers containing flammable objects may require specialpositioning), intelligent pre-fetching to a location by a work station.

These considerations may be utilized by the system to compare oneposition with another, for example, by using a weighted algorithm or anyother suitable method.

For example, when a new container is introduced into the facility, whena container is returned to the facility, when one or more containers aremoved by robots attempting to access the items stored deep within astack, when items are placed in/removed from the container, when acontainer is marked damaged, when a container is marked as dirty, etc.

This optimisation can be conducted at various times, for example, insome embodiments, the optimisation is conducted to determine a newposition for every container that has to be moved, such as those thathave to be removed so that a particular container below can be accessed.

For example, the number of robot movements and operations may bepotentially reduced if frequently ordered items and only a certainnumber of items for each given SKU# are distributed in the easiest toaccess areas of the warehouse (e.g. the closest locations to each pickstation and/or the tops of stacks of containers).

An “active window” is defined as the number of orders' worth ofcontainers to keep in the active state (on hand to service workstations)and determined by the system. The product placement optimisation module306 may be configured to assign scores to containers, biasing theoverall hive layout to tend towards a top layer of active containers,with reserve stock underneath.

This scoring system may be determined by the business rules module 310,and set based upon information such as the stock expiry date. Historicalorders may be utilized, among other information, to factor intocalculating a score for containers, and this score may be updated on acontinual basis every time the stock level of a SKU# changes. Thisscoring system may be utilized to help bias the positions of containersto maintain the optimality of positioning of containers within thefacility.

In some embodiments, the system may be configured to control robotmovement to maintain an “active” pool of one or more containers per SKU#to satisfy the demands of one or more service workstations. As an“active” pool becomes depleted, “reserve” containers may be promotedinto the “active” pool.

In some embodiments of the system, the system may be configured tocontrol robot movement using a “put-away algorithm” to determine a “bestmatch” stack location from an available pool of empty locations whenplacing a container.

In some embodiments, the “active” pool may be configured to supportparallel demand from the one or more service workstations.

In some embodiments, the product placement optimisation module 306 mayalso balance orders between the pick stations.

The robot physics model module 308 may be configured to store a set ofvariables that are designed to model the particular physical propertiesrelevant to a robot. For example, the model may indicate physicalcharacteristics such as the length, weight, height and width of a robot,the maximum carrying capacity of a robot, the rotational speed of therobot, the winch cycle time of a robot, the maximum velocity andacceleration of a robot, the ability for a robot to perform certainactions given a set amount of battery life, etc. The robot physicsmodule 308 may interface with the business rules module 310 indetermining limits on certain characteristics of robot movement,including the maximum velocity, maximum acceleration, and maximumrotational speed of a robot. For example, a robot carrying a number ofcartons of eggs may be required to only accelerate/decelerate at 25% ofthe maximum acceleration/deceleration of the robot due to thevulnerability and fragility of the eggs due to physical forces.

The business rules module 310 develops and applies a set of businessrules based upon the particular circumstances of the warehouse, robotsand communications systems. For example, the business rules module 310may provide that for certain classes of items, various restrictions arein force for the robot physics model module 308 to potentially reducethe amount of damage incurred by goods in transit. Examples of wherebusiness rules may be implemented include high risk products (e.g. acid,bleach etc.), containers with aerosols, and containers with flammablecontents, among others. Empty containers may also be treated differentlyto other containers.

The business rules may include actions such as cleaning a containerprior to re-use, slowing down robots containing certain objects, etc.

The business rules module 310 may also be configured to develop andapply sets of rules governing the placement of products. For example,different rules may be in place for high-frequency items, items that maybe picked soon due to incoming orders, etc.

The clearance module 312 may be configured to store and provideclearances for various robots. A system of clearances may be accessed todetermine whether a path is clear for a robot to traverse. The clearancemodule 312 may be implemented as a passive collision avoidance system,wherein robots are only given the smallest amount of work possiblewithout impacting performance.

Upon providing a robot with a new instruction, the clearance module 312checks that it will not be possible to collide with another robot, basedupon, for example, grid dimensions, grid positions, move commandsgenerated by planning, cancellation of move commands (generated onevents such as a controlled stop), the current positions and speeds ofrobots, braking ability of robots as well as where they have beencleared to visit.

The clearance module 312 may be configured to issue clearance “just intime”, and may be used to grant permissions to robots to continue alongtheir planned paths. A new clearance may be generated (or withheld) inresponse to each robot status report. As such, the clearance module 312may act as a path conflict resolver. Where clearances are required, theclearance module 312 may interact with the movement optimisation module304 to dynamically re-plan routes to resolve or avoid conflicts.

The clearance module 312 may provide to the control interface 302 whatthe clearances for a path would be, notification of when a clearance isissued (e.g. to the planning system as this may allow dynamicre-planning from the end of the current clearance), notification of whena clearance is withheld (e.g. to identify error cases, and to identifyneeds to re-plan), and to an alerting system (because there is apotential problem with a robot, robot communications, or the controlsystem 202).

The clearance module 312 may be configured to devise clearance schemesbased upon a set of tolerances, including missed messages, processingtime, clock sync and robot discrepancies with the physics model, amongothers.

The clearance module 312 may provide a set of safe entry times for oneor many position on the grid, based upon robot position and speedupdates, and clearances given/withheld. The set of safe entry times maybe dynamically updated as the conditions of the grid change.

The clearance module 312, in some embodiments, may be configured suchthat robots are only provided clearances for a predetermined period oftime (e.g. 3 seconds). The clearance given to a robot may be configuredsuch that the period of time is sufficient for the robot to come to restwithout risking a collision.

In some embodiments, the clearance module 312 may be configured suchthat clearances are provided such that the control system 202 may beable to miss a configurable number of status messages from the robot,and still have the robot continue operation for a short period of time.This design may result in a system that may be more tolerant to missedpackets, which may be advantageous in having continued operation evenwhere some issues with communications are encountered. The configurablenumber may be set such that there is a high probability that the controlsystem 202 is likely to receive a status message from the robot beforethe robot autonomously decelerates to be at rest at the end of itsclearance.

In some embodiments, if a robot has begun to decelerate, it will beallowed to come to rest and the control system 202 may then cancel itsonward reservations and re-plans its path over the grid.

The reservation module 314 reserves various paths on the grid (e.g.robot A is planning to take path X and reserves path X during theexpected traversal time. Robot B, knowing that robot A has reserved pathX, chooses path Y instead). The reservation module 314 may be designedto create non-conflicting robot movement plans, and may be configured towork in conjunction with the clearance module 312 and the movementoptimisation module 304.

The reservation module 314 may be configured to provide reservations fora robot for a grid location for a span of time, where no two robots maybe given overlapping reservations, taking into account tolerances forrobots being marginally off plan, tolerances for lost robotcommunication messages, and tolerances for clock discrepancies, amongothers.

In some embodiments, the reservation module 314 is used to reserveroutes in advance and to make sure that robots do not plan to takeconflicting paths, especially where there are a large number of robotactions and tasks taking place simultaneously. The reservation module314 may be configured to allow sufficient tolerance for any robot tostop under controlled braking without risking a collision.

The reservation module 314 may be configured to interact with themovement optimisation module 304 to establish the robot pathreservations sufficiently far in the future to enable forward planning.In some embodiments, the reservation module 314 and the movementoptimisation module 304 allow sufficient forward planning to completethe computation of the movement algorithms.

The command generation and scheduler module 316 generates a set ofinstructions to be transmitted to the one or more robots. Theseinstructions may include, for example, that robot A is required to moveto location B to obtain container C, bring container C to a workstationand then return container C to a particular location D. Theseinstructions may be transmitted in a near-real time/real-timeconfiguration, in a just-in-time configuration, and/or provided ahead oftime to allow for planned/scheduled routes. Further, in someembodiments, the command generation and scheduler module 316 coordinatesthe reservations and clearances to help a robot expeditiously navigateits way across a facility.

The command generation and scheduler module 316 may be configured toprovide a command set comprising a single path, or one or more paths,and/or a number of operations to be performed at various locations. Thecommand generation and scheduler module 316 provides these commands tothe robot communications module 318 to be provided to the individualrobots. In some embodiments, the command generation and scheduler module316 pre-populates instructions for a particular robot—these instructionsmay then be provided to the robot through the robot communicationsmodule 318 to be executed at a future time.

The robot communications module 318 may be configured to transmitinformation back and forth from the robots via the one or more basestations and the base station controller 206. In some embodiments, therobot communications module 318 may communicate through the use ofwireless signals. As indicated above, these instruction sets are notnecessarily just-in-time, instruction sets may be sent for thecoordination of future movements.

The robot communications module 318 may receive status reports fromvarious robots. The robot communications module 318 may be implementedin various ways, such as utilizing synchronous, asynchronous, polling,push or pull methodologies. Further, various implementations may or maynot include the use of communications “handshaking”.

In some embodiments where there is no “handshaking”, the communicationsystems may not guarantee message delivery, and lost packets may ensue.Potential advantages to such a system may include decreased bandwidthrequirements and the submission of instructions on a best efforts basis.Various schemes may be implemented to minimize the impact of lostpackets, such as timed rebroadcasting of instruction packets, sendingoverlapping instruction packets that contain parity information or othervalidation schemes, or other flow control and retransmission schemes.

In other embodiments of the invention, “handshaking” may be used toensure that packets are received.

Commands to the robots may be issued ahead of the start time for anoperation to be performed by a robot, and this time between the starttime and the issuance of a command may be a configurable parameter.

In some embodiments, commands are repeated to the robots to ensuredelivery, and the robots may provide acknowledgements that commands werereceived.

Where a message is not received before a scheduled start time, the robotmay be configured to ignore the command and may return a status messageindicating that the command was received too late. In this case, therobot control system 202 may be configured to cancel the existingreservations for the robot; and re-plan the tasking for the robot.

In some embodiments, the robot returns a regular status message thatacknowledges the last command received (for example, by a commandsequence number). The robot control system 202, in some embodiments, maybe configured such that no new command can be issued to a particularrobot until the last command issued is acknowledged by the robot. If thecommand is not acknowledged by the robot after a particular period (e.g.a configurable a time-out period); the robot control system 202 may beconfigured to cancel the existing reservations for the robot. When(command) communication is re-established with the robot; the robotcontrol system 202 re-plans the operation for the robot.

On receipt of each robot status message, the robot control system 202may be configured to extend a robot's current movement clearance throughthe clearance module 312.

The charge manager module 320 may be configured to develop a movementplan to recharge robots. The charge manager module 320 may be configuredto estimate when robots will have a specified minimum charge, and ensurethat all robots are able to charge at or before that point.

The alert/notification module 322 may be configured to provide an alertor notification to the control interface 302 when a potential issue hasarisen, or based upon a pre-determined business rule, e.g. apredetermined number of clearances have been withheld due to conflicts.

The present system and method may be practiced in various embodiments. Asuitably configured computer device, and associated communicationsnetworks, devices, software and firmware may provide a platform forenabling one or more embodiments as described above. By way of example,FIG. 1 shows a generic computer device 100 that may include a centralprocessing unit (“CPU”) 102 connected to a storage unit 104 and to arandom access memory 106. The CPU 102 may process an operating system101, application program 103, and data 123. The operating system 101,application program 103, and data 123 may be stored in storage unit 104and loaded into memory 106, as may be required. The computer device 100may further include a graphics processing unit (GPU) 122 which isoperatively connected to the CPU 102 and to memory 106 to offloadintensive image processing calculations from the CPU 102 and run thesecalculations in parallel with the CPU 102. An operator 107 may interactwith the computer device 100 using a video display 108 connected by avideo interface 105, and various input/output devices such as a keyboard115, mouse 112, and disk drive or solid state drive 114 connected by anI/O interface 109. In a known manner, the mouse 112 may be configured tocontrol movement of a cursor in the video display 108, and to operatevarious graphical user interface (GUI) controls appearing in the videodisplay 108 with a mouse button. The disk drive or solid state drive 114may be configured to accept computer readable media 116. The computerdevice 100 may form part of a network via a network interface 111,allowing the computer device 100 to communicate with other suitablyconfigured data processing systems (not shown). One or more differenttypes of sensors 135 may be used to receive input from various sources.

The present system and method may be practiced on virtually any mannerof computer device including a desktop computer, laptop computer, tabletcomputer or wireless handheld. The present system and method may also beimplemented as a computer-readable/useable medium that includes computerprogram code to enable one or more computer devices to implement each ofthe various process steps in a method in accordance with the presentinvention. In cases of more than one computer device performing theentire operation, the computer devices are networked to distribute thevarious steps of the operation. It is understood that the termscomputer-readable medium or computer useable medium comprises one ormore of any type of physical embodiment of the program code. Inparticular, the computer-readable/useable medium can comprise programcode embodied on one or more portable storage articles of manufacture(e.g. an optical disc, a magnetic disk, a tape, etc.), on one or moredata storage partitions of a computing device, such as memory associatedwith a computer and/or a storage system.

The mobile application of the present invention may be implemented as aweb service, where the mobile device includes a link for accessing theweb service, rather than a native application.

The functionality described may be implemented to any mobile platform,including the iOS™ platform, ANDROID™, WINDOWS™ or BLACKBERRY™.

It will be appreciated by those skilled in the art that other variationsof the embodiments described herein may also be practiced withoutdeparting from the scope of the invention. Other modifications aretherefore possible.

A number of different algorithms and techniques may be used indetermining a preferential path for a robot to take, including, but notlimited to: branch and bound algorithms, constraint programming, localsearch, heuristics, graph-traversal, dynamic path learning advisortechniques, pruning techniques and Bayesian graph searching techniques,Dijikstra's algorithm, Bellman-Ford algorithm, Floyd-Warshall algorithm,Johnson's algorithm, breadth-first recursive searches and depth-firstrecursive searches, weighted paths, A* search algorithm, variants on A*search algorithm (e.g. D*, Field D*, IDA*, Fringe, Fringe Saving A*,Generalized Adaptive A*, Lifelong Planning A*, Simplified Memory BoundedA*, Jump Point Search, Theta*).

In the sections below, sample search algorithms and heuristics areprovided, according to some embodiments of the invention.

Sample Search Algorithm

In this section, a sample, simplified algorithm is provided, accordingto some embodiments of the invention.

For illustrative purposes, the algorithm is provided graphically as aworkflow under FIG. 4, according to some embodiments of the invention.It is to be understood that this sample algorithm is a non-limitingexample that is solely provided as illustration to the concepts asdescribed above.

The algorithm may be an iterative search algorithm that may utilize abranch & bound search and may apply a “near-best-first” heuristic modelincluding a ‘heat map’ for congestion avoidance. Branches may beselected using a weighted cost function, and the algorithm may beloosely coupled to grid/robot shape/size.

In some embodiments, branches may be held in a sorted collection.

According to an embodiment of the algorithm, the following recursivebranch and bound function is applied:

On each iteration: (a) Select lowest cost branch b, (b) Branch b in alldirections to create new branches B (c) For each branch b′ in B: If b′has reached target: return b′ Else: Add b′ to search Cost comparatorallows tracking of lowest cost branch for next iteration

In performing a path search, various heuristics may be applied to reducethe computations required, by, depending on the heuristic applied,removing entire branches or conducting a less computationally intensiveanalysis. Sample heuristic techniques are provided in a later section ofthis specification.

Where a new branch has a conflict with a path that another robot may betaking, or conflicts with an idle robot, the search algorithm may:

-   -   Alter the branch to outrun the conflicting reservation (if        acceleration profile allows);    -   Alter the branch to contain a wait at any position along its        path; or    -   Discard the branch as infeasible In some embodiments, paths may        be selected where robots wait at their starting points.

In doing so, the search space can never truly be exhausted (e.g. ifthere are currently no acceptable paths for a robot to take to itsdestination, a path may be selected where the robot waits until anon-conflicting path is available).

Sample Heuristic Techniques

The search algorithm may be configured to preferentially balance anynumber of goals such as taking the shortest possible time, tending toavoid congested areas etc.

The following provides a simple, non-limiting example of the applicationof the use of heuristics for illustrative purposes.

In an embodiment of the invention, each search may track the currentlowest cost branch and weighted cost functions may be used to bias theselection ordering of the branches based upon various heuristics, whichmay include: (a) a projected shortest path time and (b) a projectedaccumulated heat based on a heat-map. Other heuristic techniques may becontemplated but for illustrative purposes, further detail will beprovided for the two identified above.

(a) Projected Shortest Path Time

For any branch, the control system may determine the shortest possiblepath time from the current branch tip to the destination.

The projected time cost may then be determined as the total time for thebranch so far (including, for example, any waiting that was required,etc.) added to the time for the shortest possible unconstrained path tothe target.

(b) Projected Minimum Heat (Based on a Heat Map)

In developing a heat-map, a ‘heat’ value may be assigned to eachcoordinate, approximating a model of congestion at points on the grid,or for particular areas of the grid.

FIG. 5 provides a sample heat map, according to some embodiments of theinvention.

In some embodiments, the ‘heat’ value may be determined using proximityto workstations, but in other embodiments of the invention, the ‘heat’may be observed/learned/calculated/predicted using a variety of othertechniques, some of which may be dynamic or iterative techniques.

Similar to the projected shortest path, an unconstrained path may beprojected to the destination.

The projected minimum heat may then be determined. In some embodiments,the projected heat cost is the sum of the heat of all visitedcoordinates in the current branch added to the heat of the ‘coldest’(the least hot) of the projected paths.

Sample Weighted Cost Function

In some embodiments, the algorithms used are based upon weighted costfunctions. Such algorithms may be amenable to optimisation of theassociated cost coefficients by studying the results of large numbers ofconcurrent simulations in the cloud configured to use differentcoefficients, and/or applying various machine learning approaches andtechniques, possibly using large sets of observed and/or simulated data.

In some embodiments, the search algorithm has two cost coefficients: (a)the projected shortest path time coefficient (C_(t)), and (b), theprojected minimum heat coefficient (C_(h)).

In some embodiments, the search algorithm may include the followingequation:

Branch cost=C _(t)*PSP+C _(h)*PMH

(Where PSP refers to projected shortest path time and PMH refers toprojected minimum heat)

In some embodiments, the cost function may be utilized with configurableor machine-learning derived exponents that model complex relationships.A sample, simplified cost function, provided for illustrative purposesincludes:

Branch cost=C_(t)*PSP^(x)+C_(h)*PMH^(Y), where x & y may be furtherconfigurable and/or machine learned exponents.

FIG. 6a provides a table demonstrating how a search branch's projectedcost could change as a result of using different coefficients in thepath search algorithm. These coefficients could be derived usingrefinement by a machine learning technique according to some embodimentsof the invention.

FIG. 6b illustrates how the use of different cost coefficientsdemonstrated in FIG. 6a changes which branch will be selected by thenext iteration of the search algorithm.

In some embodiments, the control system 202 may be further configured todevelop, adapt and apply a set of rules over time to refine a set ofmachine learned coefficients and/or exponents. The use of machinelearned coefficients and/or exponents may potentially increase theeffectiveness of the heuristic techniques over a duration of time.

Sample Shunting Search

The system may be configured to adjust robot paths to take into accountthe positions of idle robots. In some embodiments, there may be idlerobots which may be tracked independently of robots with tasks. Theseidle robots may not have planned paths and associated reservations andmay need to be considered separately.

A separate “shunt search” may be performed when a task is beingfinalised. The “shunt search” may be comprised of finding paths to moverobots which are idle now, or will go idle in the path of the robotwhich is being tasked (hereinafter referred to as the primary robot), tolocations where they may continue being idle and not be in the way ofthe path of the primary robot.

This “shunt search”, in some embodiments, comprises performing a searchwhere, for each robot which is idle now, or will go idle in the path ofa primary robot, a search is performed which may be deemed solved uponfinding a location in which it can remain indefinitely.

The “shunt search” may use the same branch & bound search algorithm asthe primary robot path search, but may have different cost coefficientsand solution criteria. If a robot is unable to move out of the way intime, a wait may be added to the start of the primary robot's path andthe primary robot's path may be recalculated.

Sample Put-Away Algorithm

An algorithm may be used to determine a stack location for a containerto be returned to. Containers may be returned for various reasons, andthe location in which a container is returned to may be optimised forvarious advantages, such as improving the distribution ofobjects/containers in the hive.

Every stack location in the hive may be scored with a configurable,weighted cost function of:

-   -   Average distance (measured in robot operation time) from all        workstations;    -   Distance (measured in robot operation time) from closest        workstation; and    -   Approximate dig cost (if depth>0)

The system may keep a “Hive Plan” of the current end state of the hiveafter all operations in the plan have been executed.

The “Hive Plan” may also track the “available surface” in which a robotcan place a container. Each container has an index of its position inthe totally ordered set of containers, as defined by the productplacement optimisation module 306.

Each stack location has an equivalent index in the totally ordered setof stack locations as defined by the weighted cost function.

These indices are remapped to the range 0-1 by dividing by the size oftheir respective sets, and the stack locations in the available surfaceare ranked by how closely their indices match that of the container'sindex.

The final selection is made via a weighted cost function of thedifference between these indices and other factors such as how long theideal path is from the source to the stack location and how long thestack is reserved for in the current plan.

Other business rules can be enforced at this stage, such as limiting thetotal weight of a stack; controlling the position of hazardous orspecial substances (e.g. aerosols and inflammable materials) etc.

Sample Return Scenario

The following provides a sample returns process that may besupported/controlled by the system. This process could be applied wherean order is scratched, totes did not leave the hive, an order isreturned by customer, or a delivery failed to a customer. Othersituations may also be contemplated. A returned product (which may be ina container, other holding device such as a tote, etc.) can be processedat a workstation that provides reworking or rejection of the products.

The container/tote may be scanned so that the controller can positionthe storage bins it expects to need close to the workstation. Supplybins may be selected based on SKU and expiry date. Product items may beremoved by the picker one-by-one and scanned. When the container arrivesat the workstation, a picker (automated or manual) may beinstructed/controlled to place the item into the container.

The picker may also be asked to confirm there are no further eaches ofthis SKU# left, before the container is released.

Products that are no longer suitable for return to stock may be pickedinto containers; which, at various times, such as when full, or at theend of the day, may be removed at the workstation and the contents sentto another area, such as the staff shop or disposal as appropriate.

While the disclosure has been provided and illustrated in connectionwith specific, presently-preferred embodiments, many variations andmodifications may be made without departing from the spirit and scope ofthe invention(s) disclosed herein. The disclosure and invention(s) aretherefore not to be limited to the exact components or details ofmethodology or construction set forth above. Except to the extentnecessary or inherent in the processes themselves, no particular orderto steps or stages of methods or processes described in this disclosure,including the Figures, is intended or implied. In many cases the orderof process steps may be varied without changing the purpose, effect, orimport of the methods described. The scope of the invention is to bedefined solely by the appended claims, giving due consideration to thedoctrine of equivalents and related doctrines.

1. A system for controlling movement of transporting devices arranged totransport containers, the containers being stored in stacks arranged ina facility, the facility having pathways arranged in a grid-likestructure above the stacks, the transporting devices being configured tooperate on the grid-like structure, the system comprising: a movementoptimisation unit configured to determine a route of a transportingdevice from one location on a grid-like structure to another location onthe grid-like structure for each transporting device; a reservation unitconfigured to reserve a path on the grid-like structure for eachtransporting device based on the determined route, wherein the pathreserved for each transporting device is provided such that no twotransporting devices have locations on the grid-like structure whichwould cause transporting devices to overlap at a same time; and aclearance unit configured to provide clearance for each transportingdevice to traverse a portion of the reserved path.
 2. The systemaccording to claim 1, wherein the clearance unit is configured to grantor withhold providing clearance for a transporting device to traverse aportion of the reserved path in response to a status report receivedfrom each transporting device.
 3. The system according to claim 1,wherein the clearance unit is configured as a passive collisionavoidance system, wherein each transporting device is operated in amanner without impacting system performance.
 4. The system according toclaim 1, wherein the clearance unit is configured to grant or withholdproviding clearance for a transporting device to traverse a portion ofthe reserved path based on at least one of: dimensions of the grid-likestructure, positions on the grid-like structure, commands to move thetransporting device, cancellation of commands to move the transportingdevice, a current position of the transporting device, a speed of thetransporting device, a braking ability of the transporting device, andclearances provided to the transporting device.
 5. The system accordingto claim 1, wherein at least one of the clearance unit, and the movementoptimisation unit is configured to dynamically re-plan a route of atleast one transporting device.
 6. The system according to claim 1, whenat least one message is not received by the system from the transportingdevice, the movement optimisation unit is configured to calculate aroute for a transporting device that traverses a portion of the reservedpath to resolve or avoid conflicts.
 7. The system according to claim 1,wherein the clearance unit is configured to provide clearance based upona set of tolerances, including at least one of: missed messages, aprocessing time, a clock sync, and transporting device discrepancieswith a physics model.
 8. The system according to claim 1, wherein theclearance unit is configured to calculate a set of safe entry times forat least one position on the grid-like structure based upon at least oneof transporting device position, speed updates, and clearances given andwithheld.
 9. The system according to claim 1, wherein the clearance unitis configured to provide clearances for a predetermined period of time.10. The system according to claim 1, comprising: a control unitconfigured to control movement of the transporting devices.
 11. Thesystem according to claim 10, wherein the clearance unit is configuredto provide to the control unit at least one of: clearances required totraverse a reserved path, notification of when a clearance is issued,notification of when a clearance is withheld, and problems with atransporting device, wherein the control unit is configured to controlmovements of the transporting devices based on the information receivedfrom the clearance unit.
 12. A storage system comprising: a first set ofparallel rails or tracks extending in a first direction, and a secondset of parallel rails or tracks extending in a second directiontransverse to the first set in a substantially horizontal plane to forma grid pattern having grid spaces; stacks of containers located beneaththe rails that are arranged such that each stack is located within afootprint of a single grid space; load handling devices, each loadhandling device being arranged to selectively move laterally in thefirst and second directions, above the stacks on the rails; and thesystem according to claim
 1. 13. The storage system according to claim12, wherein each load handling device has a footprint that occupies asingle grid space in the storage system, such that a load handlingdevice occupying one grid space does not obstruct a load handling deviceoccupying or traversing the adjacent grid spaces in the first and seconddirections.
 14. A method for controlling movement of transportingdevices arranged to transport containers, the containers being stored instacks arranged in a facility, the facility having pathways arranged ina grid-like structure above the stacks, the transporting devicesconfigured to operate on the grid-like structure, the method comprising:determining a route from one location on the grid-like structure toanother location on the grid-like structure for each transportingdevice; reserving a path on the grid-like structure for each of thetransporting devices based on the determined route, wherein the pathreserved for each transporting device is provided such that no twotransporting devices have locations on the grid-like structure whichwould cause transporting devices to overlap at a same time; andproviding clearance for each transporting device to traverse a portionof the reserved path.
 15. The method according to claim 14, wherein theproviding clearance comprises: granting or withholding clearance for atransporting device to traverse a portion of the reserved path inresponse to a status report received from each transporting device.